Microled based invisible illumination for eye tracking

ABSTRACT

Eye-tracking systems and methods utilize transparent illumination structures having a plurality of IR μLEDs distributed with a predetermined pattern within the transparent viewing area of illumination structures. The μLEDs are small enough that they are not visible by a user during use of an HMD or other mixed-reality device, for example, such that they can be positioned within the line-of-sight of the user through the illumination structure and without visibly obscuring or interfering with the user&#39;s view of the mixed-reality environment by the mixed-reality device.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.17/830,672 filed on Jun. 2, 2022, entitled “MICROLED BASED INVISIBLEILLUMINATION FOR EYE TRACKING,” which issued as U.S. Pat. No. 11,579,444on Feb. 14, 2023, and which application is expressly incorporated hereinby reference in its entirety.

BACKGROUND

Mixed-reality systems, including virtual-reality and augmented-realitysystems, have recently received significant interest for their abilityto create immersive experiences for users. Conventionalaugmented-reality (AR) systems create an augmented reality scenario bygenerating holograms that are rendered in the user's line-of-sight toobjects in the real world. In contrast, conventional virtual-reality(VR) systems create a more immersive experience because a user's entireview is obstructed by a virtual world.

As used herein, AR and VR systems are described and referencedinterchangeably using the umbrella term “mixed-reality system(s).”Unless specifically stated or unless specifically required, asunderstood by one of skill in the art, the descriptions herein applyequally to any and all types of mixed-reality systems, including ARsystems, VR systems, and/or any other similar system capable ofdisplaying virtual objects to a user. Accordingly, from this pointforward, the disclosure will use the term mixed-reality system todescribe any of the systems referenced above.

Of note, many mixed-reality systems use one or more on-body devices,such as a head-mounted display (hereinafter “HMD”), to render a virtualenvironment for a user. Continued advances in hardware capabilities andrendering technologies have greatly increased the realism of virtualobjects displayed within a mixed-reality environments, particularly withthe use of HMDs. For example, as the user moves their head during amixed-reality session, the rendered mixed-reality environment isautomatically updated, with the holograms being repositioned, relativeto the user's movement, and such that the user is provided with a properperspective and view of the virtual objects in the mixed-realityenvironment.

Recent advances in this technology space also relate to the use of eyetracking systems to track a movement of the user's eyes. As a result, amixed-reality system can also respond to a user's eye movements, inaddition to their general head and body movements. In particular, bytracking a user's eye movements and the direction of the user's gaze, itis possible to position the holograms rendered by the user's HMD in sucha manner that they are properly positioned in the mixed-realityenvironment relative to the user's gaze.

One technique for tracking eye movement and for determining thedirectionality of a user's gaze includes the analysis of infrared (IR)light signals that are directed towards and reflected off of a user'seyes. For instance, an HMD can be equipped with one or more IR lightsources that emit IR light at the user's eyes from different directions.The HMD is also equipped with IR camera(s)/sensor(s) that are positionedat known locations relative to the IR light sources. The IR light thatis reflected off of the user's eyes, referred to herein as glints, canbe detected by the sensors and used to determine the XYZ location andgaze direction of the user's eyes, which correspond with the user's gazedirection in the mixed-reality environment. This is possible, forexample, by analyzing position of the glints relative to the positioningof the pupil center of the eye.

To enhance the accuracy and processing performed by IR eye trackingsystems, it is often desirable to position several different IR lightsources around the user's eyes to create different glints off of thecornea. In some instances, multiple IR cameras/sensors can also be usedfor each eye, although this is not necessarily required.

Unfortunately, conventional mixed-reality systems are somewhat limitedin regard to where the IR light sources can be positioned. Inparticular, conventional systems have historically positioned the IRlight sources along the peripheral rims or other mounting structures ofthe HMD where the actual display lenses and screens are mounted. Thisperipheral placement of the IR light sources is not always optimal andcan require the utilization of more light sources and/or larger and morepowerful light sources than would otherwise be required if the lightsources could be positioned closer or more optimally relative to theiris of the user's eyes.

Some AR systems have attempted to position IR light sources away fromthe peripheral rim of the display screen, closer to the user's eyes andwithin the user's field of view. However, these types of existingsystems are problematic in that they can create visual obstructions tothe user's perspective of the mixed-reality environment. In particular,existing IR light sources are typically sized in the 1.0 mm to 4.0 mmdiameter/width range. Such sizes are very noticeable, particularly whenpositioned within the user's field of view. For at least this reasonmost conventional systems have only positioned the IR light sources in asuboptimal location along the peripheral rim of the lenses/displays.

In view of at least the foregoing issues, there is an ongoing need anddesire for improved systems and techniques for performing IR eyetracking in mixed-reality devices, particularly for AR devices.

The subject matter claimed herein is not limited to embodiments thatsolve any disadvantages or that operate only in environments such asthose described above. Rather, this background is only provided toillustrate one exemplary technology area where some embodimentsdescribed herein may be practiced.

BRIEF SUMMARY

The disclosed embodiments include eye-tracking systems and methods, suchas, but not limited to improved systems and techniques for performing IReye tracking in mixed-reality devices, particularly for AR devices, andin a manner that is not constrained by the form factor of the rims andother structures used to mount the lenses/displays used to render themixed-reality environment.

In some instances eye tracking systems are configured with displaycomponents, such as illumination lenses, or other display components(powered or unpowered) having IR micro-LEDs (also referred to herein asmicroLEDs, mLEDs, or μLEDs), which are embedded within or disposeddirectly on top of the surfaces of the display component (e.g., lens orother display component). In such embodiments, the IR μLEDs arepositioned within the peripheral boundary of the rims or otherstructures that are used to mount the display component to the HMD/ARdevices.

The size of the IR μLEDs, with a range from about 10 um (square) to abut100 um (square) enables their positioning directly in front of a user'sfield of view or line-of-sight, and without being visibly perceived orobstructive to the user's view of the mixed-reality environment. As aresult, the IR μLEDs are essentially invisible to the user during use,while also providing sufficient illumination and optimal placement forfacilitating tracking of the user's eye movements during use.

Some embodiments also include HMDs or other mixed-reality devices (e.g.,AR/VR devices) that include lenses/displays having a plurality of μLEDsthat are configured to emit light in the IR spectrum for performing eyetracking by the mixed-reality devices and which are embedded directlyinto the lenses/displays of the mixed-reality devices that are used torender the virtual objects presented in the mixed-reality environments.In these embodiments, the positioning of the IR μLEDs on the HMD arebeneficially not constrained to the particular form factor of the rimsor other structures that are used to mount the lenses and/or otherdisplay component used for rendering the mixed-reality environment.

Some embodiments are also directed to the actual structures that areused for assembling the lenses or other display components that areconfigured for being positioned directly in front of a user's eyesduring use within an HMD or other mixed-reality device and that arecomposed of a substantially transparent substrate. This substrate isconfigured with a plurality of IR μLEDs affixed to and/or embeddedwithin the viewing area of the illumination structures. Theseillumination structures can be formed into different sizes and shapesfor assembly into different types of user devices, such as, but notlimited to mixed-reality devices.

Some embodiments also directed to the processes for manufacturing lenseswith a plurality of IR μLEDs affixed to and/or embedded within thelenses, as well as processes for manufacturing eye tracking devices,such as, but not limited to mixed-reality devices, which incorporate thelenses having the IR μLEDs.

Some embodiments are also directed to the methods and systems used forperforming eye tracking based on IR light that is generated by IR μLEDsaffixed to and/or embedded within lenses of a mixed-reality device.These processes include detecting and measuring IR light which isreflected off of a user's eye(s) after being generated by the IR μLEDs.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

Additional features and advantages will be set forth in the descriptionwhich follows, and in part will be obvious from the description, or maybe learned by the practice of the teachings herein. Features andadvantages of the embodiments may be realized and obtained by means ofthe instruments and combinations particularly pointed out in theappended claims. Features of the present embodiments will become morefully apparent from the following description and appended claims or maybe learned by the practice of the embodiments as set forth hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to describe the manner in which the above-recited and otheradvantages and features can be obtained, a more particular descriptionof various embodiments will be rendered by reference to the appendeddrawings. Understanding that these drawings depict only sampleembodiments and are not therefore to be considered to be limiting of thescope of the invention, the embodiments will be described and explainedwith additional specificity and detail through the use of theaccompanying drawings in which:

FIG. 1 illustrates an example of an HMD.

FIG. 2 illustrates a representation of IR light reflecting off of a usereye with specular and scatter reflections.

FIG. 3 illustrates a representation of a substrate or transparentstructure that is operationally configured with a plurality of IR μLEDsthat are distributed in a grid pattern.

FIGS. 4A-4D illustrate additional representations of substrates or othertransparent structures that are operationally configured with aplurality of IR μLEDs having different distribution patterns.

FIG. 5 illustrates a representation of a HMD configured with a displaycomponent having a plurality of IR μLEDs embedded within or on top ofthe display component, within the peripheral boundary of the mountingrim and structures used to secure the display component to the HMD.

FIG. 6 illustrates a representation of a lens assembly for use with adevice configured to perform eye tracking and that includes a pluralityof IR μLEDs embedded within or on top of the lens, positioned to bewithin the line-of-sight of a user when the lens assembly is positionedin front of the user's eyes during use.

FIG. 7 illustrates a representation of a visor assembly for use with adevice configured to perform eye tracking and that includes a pluralityof IR μLEDs embedded within or on top of the transparent visor,positioned to be within the line-of-sight of a user when the visor isworn by a user eyes during use.

FIG. 8 illustrates a representation of a manufacturing process flow formanufacturing a substrate or transparent structure with a plurality ofIR μLEDs.

FIG. 9 illustrates a flow chart that includes acts associated withmethods for manufacturing a substrate or transparent structure having aplurality of IR μLEDs embedded within or on top of the planar surface ofthe transparent structure.

FIG. 10 illustrates a flow chart that includes acts associated withconfiguring systems to perform eye tracking based on IR light emittedfrom a transparent illumination structure having a plurality of IR μLEDsdistributed within the transparent viewing area of illuminationstructure.

FIG. 11 illustrates an example system that may be used to track a user'seye movements based on IR light reflected off of the user's eyes inresponse to IR light being emitted from IR μLEDs embedded within or ontop of the transparent structures positioned in front the user's eyes.

DETAILED DESCRIPTION

Disclosed embodiments comprise and/or utilize transparent illuminationstructures having a plurality of IR μLEDs distributed within thetransparent viewing area of illumination lens structures. In someembodiments, the transparent illumination structures are incorporatedinto HMD devices that perform eye tracking based on light emitted fromthe IR μLEDs.

With regard to the embodiments that describe or perform eye tracking, itwill be appreciated that this eye tracking can be performed for a user'sleft eye, a user's right eye, or a combination of the user's left andright eyes. Therefore, the embodiments are not limited to tracking onlya single eye, nor do they necessarily require tracking movements of botheyes. Additionally, for brevity, the disclosure will (from this pointforward) present examples related to only a single eye. These examplesare for illustrative purposes only, and it will be appreciated that theprinciples may equally be applied to scenarios involving more than oneeye.

With regard to the term lens, which is used throughout this disclosure,it will be appreciated that the term lens can be broadly interpreted toinclude both powered and unpowered configurations. For instance, a lenscan include display components that are configured with optical power toperform an optical function in a mixed-reality application (e.g.,filtering, displaying, etc.). Additionally, the term lens should also bebroadly interpreted to include entirely passive structures, suchzero-power transparent materials. In either instance, the lens isconfigured for being positioned in front of a user's eyes during use ofa mixed-reality device and through which a user can visually perceivereal-world objects, including other display components of an HMD. Inmost embodiments, the lens is a substantially planer or flat structureon which the referenced micro-LEDs can be positioned (on either side ofthe lens and/or within the lens material).

As previously mentioned, the disclosed embodiments may be implemented toovercome many of the technical difficulties and constraints associatedwith tracking a user's eye, and particularly with regard to tracking auser's eye movements when using an HMD. In particular, the disclosedembodiments enable IR light sources to be positioned directly within theviewing area of the HMD lenses, at optimal orientations relative to theuser's eye/iris and corresponding camera sensors, without obstructingthe user's view of the mixed-reality environment, and withoutconstraining the positioning of the IR light sources to the rims orother HMD structures positioned outside the periphery of the lensviewing area. This is possible, according to the disclosed embodiments,by utilizing illumination lens structures having IR μLEDs that aresmaller than 100 μm in any given direction. These micro-LEDs are notperceptible by a user, even though they are embedded within the lensesof the HMD, and do not, therefore, obstruct or interfere with thepresentation of the mixed-reality environment by the HMD during use.

Attention will now be directed to FIG. 1 , which illustrates arepresentation of an HMD 100. In this embodiment, the HMD 100 includesone or more lenses (110, 120) that are used to provide different opticalbenefits to the user of the HMD. One such benefit can be thedisplay/rendering of holograms or projections that are perceived by theuser to exist in their environment. Another benefit that can be providedby the lenses includes filtering light that exists in the ambientenvironment.

In some instances, the HMD is also configured to perform eye tracking,based on detected light reflections (glints) that are captured by acamera 130 or other sensor (e.g., a silicon photomultiplier (SiPM)sensor or other type of sensor). For instance, during use, light isemitted from one or more light sources (e.g., IR LEDs 140, of which onlyfour are called out), which may surround the user's eye. After the lightis emitted, glints are reflected off of a user's eye (particularly theuser's iris) and detected by the camera 130. Depending on theintensity/strength of the light that is perceived, relative tosource/timing of the light being emitted, the HMD light processingmodule can detect the positioning (relative location and orientation) ofthe user's eye/iris.

Additional processing of imagery captured by the system cameras/sensorscan also be used to distinguish the user's pupil from the user's iris.Such imagery can help the system map the location of the user's eye andorientation/gaze of the user's eye relative to a projected hologram orother object, for instance, to detect user attention/focus. Knowing theuser's eye positioning can also be used by the system to position andreproject holograms within the mixed-reality environment at desiredlocations relative to the user's visual perspective of the mixed-realityenvironment.

FIG. 2 illustrates a representation of a user's eye 200 in which an IRlight source 210 (e.g., an IR LED) is emitting IR light 220 towards theuser's eye 200. The IR light 220 is reflected back as both specularreflections 230 and as scatter reflections 240. This illustration alsoshows how a camera 250 or other sensor (generally referred to herein asa camera) can be positioned to detect one or more of the reflections.

By knowing the location of the light source(s), and the timing foremitting the light from the light source(s), as well as the location ofthe camera(s) and the measured intensity and timing of the detectedlight reflections that are reflected off of a user's eyes, a system canascertain the relative positioning (location/orientation) of the user'seye/iris. This is possible, in part, because light reflects differentlyoff of different portions of the user's eyes (e.g., it reflectsdifferently off of the pupil and iris area of the cornea than thesclera). These differences are detected and measured, in part, based onwhether the reflections are specular or scatter reflections.

More detail will not be given at this time, regarding the measuring andprocessing of reflected light signals for identifying the position of auser's eye, inasmuch as this type of eye tracking is well understood tothose of skill in the art.

However, with regard to the structures used to perform eye tracking,specifically the IR light sources, it will be noted that it would bedesirable, in some instances, to position the light sources (e.g., lightsources 140 at a location within the rim 150 or other mountingstructures of the HMD), so that the light sources can be positioned moreoptimally and closer to the user's cornea, for example.

For instance, it would be desirable to position the light sources, insome instances, off of the rim 150 and directly into or on the lensesthat the user looks through, even though they would be within the user'sline-of-sight that passes through the lens area contained within theperipheral edge of the lens 110. Unfortunately, conventional IR lightsources are too large (e.g., 1-4 mm) to position within a user'sline-of-sight without causing obstructions to the user's view of theirenvironment viewed through the lens 110.

To help address these problems, the current embodiments includeillumination lenses configured with IR μLEDs that are distributed withinthe bounded lens area and user's line-of-sight. With theseconfigurations, it is possible to optimally place the light sourcesproximate the user's eye and without having to account for the existingconstraints imposed by the physical form factors of the HMD mountingstructures.

FIG. 3 illustrates one representation of an IR μLED lens structure 300having sixteen IR μLEDs (labelled as LED1, 2, 3, 4, 5, 6, 7, 8, 9, 10,11, 12, 13, 14, 15, LED16) arranged in a grid pattern. The IR μLEDs arepositioned on transparent substrate 310, along conductive traces 320that form an electric circuit between an anode terminal and a cathodeterminal. When this circuit is powered by a power source of the HMD, forexample, which is not shown but which can be electrically connected tothe anode and cathode terminals and controlled by an illuminationcontrol unit, the IR μLEDs will be activated and emit IR light.

The wavelength of light that is emitted from the IR μLEDs falls withinthe range of between about 790 μm and 1 mm. In some preferredembodiments, the IR μLEDs emit light at a wavelength of about 800-900 μmand even more preferably about 850 μm.

While the grid pattern shown is one possible distribution of IR μLEDs,it will be appreciated that there are many different layouts andpatterns that may be used for distributing the IR μLEDs on/within theillumination lens structures described herein. It will also beappreciated that it is possible for different IR μLEDs in the set ofdistributed IR μLEDs to each emit the same and/or different spectrums oflight. By emitting different spectrums, for instance, it is possible toidentify the source of the light more easily when measuring the lightreflections. Furthermore, while all of the IR μLEDs may be contained ona single circuit, as shown, it is also possible to position different IRμLEDs on different circuits that are electrically insulated from eachother (not presently shown), so as to enable selective control over theillumination by the different IR μLEDs at different known moments intime. This can also help the system identify the location of differentIR μLEDs that are activated at any particular time.

To further illustrate the capability of modifying the distribution ofthe IR μLEDs on the illumination lens structures, some additionalnon-limiting examples will now be provided.

For instance, as shown in FIGS. 4A-4D, the quantity of IR μLEDspresented on the different illumination lens structures, and theiroverall distribution patterns are varied. Even more particularly, thedistribution pattern of the IR μLEDs (shown as small rectangular boxes)is somewhat circular/oval in FIGS. 4A, but somewhat rectangular in FIG.4C, and somewhat diamond-shaped in FIG. 4D. The embodiment shown in FIG.4B, on the other hand, includes two different patterns, an outer patternthat is circular/oval and an inner pattern that is triangular.

Additionally, while the illumination lens structure 400A comprises six(6) IR μLEDs, the illumination lens structures 400C and 400D eachcomprise eight (8) IR μLEDs. The illumination lens structure 400Bcontains nineteen (19) IR μLEDs.

The different quantities and patterns of IR μLEDs can be varied toaccommodate different needs and preferences, including, but not limitedto varied needs to illuminate larger or smaller areas, to illuminatemore intensely for certain ambient environments and use scenarios, toilluminate from further distances to accommodate different lens formfactors, to consume less power, to perform more precisetracking/measurements, etc.).

To further illustrate the possible variations in which the IR μLEDs canbe distributed on/within the illumination lens structures, attention isdirected to FIGS. 5-7 . These illustrations show representations of theillumination lens structures embodied within an HMD or other device thatis configured to perform eye tracking. These HMD devices can comprise,for example, mixed-reality devices (AR and/or VR devices) that areconfigured to track user eye movements and to render holograms in amixed-reality environment based at least in part on the detectedpositioning of the user's eye(s).

In FIG. 5 , an HMD 500 is configured with a transparent illuminationlens structure 510 that has a plurality of IR μLEDs 540 (only two ofwhich are specifically called out). As shown, the IR μLEDs 540 aredistributed in a circular pattern, eight (8) for each eye and lens area.The different IR μLEDs 540 can optionally be connected on a singlecircuit or two or more different circuits. The electrical traces formingthe circuit are not shown. Light emitted from the IR μLEDs will bedirected at least partially towards the user's eye(s) during use and atleast some of that light will be reflected back and detected by thedevice camera 530.

FIG. 6 shows another embodiment of a pair of glasses 600 having a firsttransparent illumination lens structure 610 that includes four (4) IRμLEDs and a second transparent illumination lens structure 620 thatincludes eight (8) IR μLEDs. This example is used to illustrate that itis not essential for both sides of an HMD to have matching/symmetricdistribution of the IR μLEDs. Regardless of the quantity anddistribution of the IR μLEDs, it will be appreciated that the lightemitted from the IR μLEDs will be directed at least partially towardsthe user's eye(s) during use and at least some of that light will bereflected back and detected by the device camera(s) 630. Additionally,the different IR μLEDs 640 can optionally be connected on a singlecircuit or two or more different circuits. The electrical traces formingthe circuit are not shown.

In a related embodiment, the HMD system 700 shown in FIG. 7 includes atransparent illumination lens structure 710 having the generalconfiguration of a visor. In this example, one side of the visor hasseveral IR μLEDs forming a circular pattern (only a few of the IR μLEDs740 are specifically called out). The other side of the visor has four(4) IR μLEDs distributed in a square pattern. In other embodiments (notshown), both sides of the visor have a same quantity of IR μLEDs and/ordistribution pattern of the IR μLEDs. Regardless of the quantity anddistribution of the IR μLEDs, it will be appreciated that the lightemitted from the IR μLEDs 740 will be directed at least partiallytowards the user's eye(s) during use and at least some of that lightwill be reflected back and detected by the device camera(s) 730.Additionally, the different IR μLEDs 740 can optionally be connected ona single circuit or two or more different circuits. The electricaltraces forming the circuit are not shown.

With regard to all of the foregoing examples, it will be appreciatedthat the IR μLEDs are not shown to scale. In fact, to the contrary, theIR μLEDs are so small (<100 μm), as are the thicknesses of the traces,that they would not be perceptible on the current illustrations if theywere represented at scale. This is beneficial, however, for the IR LEDsto be so small, so they can be used to illuminate a user's eye with IRlight, while being positioned within/on the lens that the user looksthrough and without obstructing the user's view through the lens.

The electrically conductive traces are also very thin, having a width of<50 μm or even <25 μm, such that they are visually unnoticeable andessentially invisible to the user during use with close proximity to theuser's eyes. This configuration is particularly beneficial for enablingthe traces to be positioned within or on the illumination lensstructures, within the user's line-of-sight through the lenses, andwithout obstructing the user's view of the environment perceived throughthe lens during use, even if they are positioned directly in front ofthe user's eye.

Attention will now be directed to FIG. 8 . This illustration visualizesa manufacturing flow for manufacturing the illumination lens structuresdescribed herein, which include IR μLEDs embedded within or otherwisedistributed on the illumination lens structures, within the areacircumscribed by the edge boundaries of the illumination lensstructures.

As shown, the manufacturing process includes obtaining a wafer 810comprising one or more IR μLEDs, although only a single IR μLED (815) isspecifically singled out on the wafer 810. This wafer 810 may be an epiwafer or epitaxy wafer, for example, which is formed through an epitaxygrowth or deposition process. In some instances, the wafer comprises asurface area containing tens, hundreds or even thousands of GaAs-basedIR microLEDs that are formed on the wafer with an epitaxy growth.

These IR μLEDs are extractable and transferable to a substrate, forexample, by a laser lift-off or an elastomeric transfer. Detailsregarding laser lift-off processes and elastomeric transfer processeswill not be described at this time, as such processes are known and usedfor extracting/transferring other types of LEDs from epi wafers. Suchprocesses can also be used for extracting/transferring the IR μLEDsdescribed herein.

The process 800 represented in FIG. 8 also includes obtaining thesubstrate 820 to transfer the IR μLEDs onto. As shown, a transferprocess 830 (e.g., laser lift-off or elastomeric transfer) is performedto transfer one or more of the IR μLEDs onto electrically conductivetraces 825 that are already positioned on the substrate 820 and thatform one or more different circuits 827 on the substrate.

The sizes of the IR μLED that are removed from and positioned on thesubstrate are each constrained to <100 μm in any direction (e.g., width,length, and height), such that the maximum dimension of any measurablelength across any portion of the IR μLEDs, in some embodiments, evencorner to corner is <100 μm. In some instances, the maximum sizedimension of the IR μLEDs is <75 μm, or <50 μm, or even <20 μm. In someembodiments the maximum size dimension of the IR μLEDs is about 10 μm.

The width of the traces is also similarly constrained, so as to not havea thickness of greater than <50 μm, <40 μm, <30 μm, or even <20 μm. Insome instances, the width of the traces is about 20 μm. In this regard,it will be appreciated that the width of the traces may vary toachieve/control a desired impedance of the traces. The traces arepreferably composed of a conductive metal, such as Ag, Cu or Al.

In some embodiments, the substrate 820 comprises a transparent PCBceramic or glass structure. In other instances, the substrate 820 iscomposed of a flexible transparent plastic material (e.g., PET thinfilm). The thickness of the substrate 820 can vary to accommodatedifferent needs and preferences. In some instances, the substrate 820has a thickness in a range of between 0.05 mm and 0.2 mm, and preferablyabout 0.1 mm. The substrate 820 is also referred to herein as abackplane.

After the illumination lens structure or IR wafer assembly 840 isconfigured, comprising the composite assembly of the IR μLEDstransferred to the substrate and corresponding traces/circuits on thewafer, one or more segments/portions of the IR wafer assembly 840 can becut out or separated from the rest of the assembly 840 as stand-aloneillumination lens structures 850. Optionally, before or after separatingthe discrete illumination lens structures 850 from the rest of theassembly 840, the illumination lens structure 850 can be coated with aprotective coating 860 and/or laminated with one or more other lensmaterials to provide special protection and/or optical properties to theIR μLEDs and illumination lens structure 850.

Attention will now be directed to FIG. 9 , which illustrates a flowdiagram 900 of acts associated with methods for configuring an HMDdevice to perform eye tracking, wherein the HMD includes an illuminationlens that contains a plurality of IR μLEDs, and each IR μLED of theplurality of IR μLEDs has a maximum size dimension of <100 μm.

The first illustrated act includes the configuring of the HMD with theillumination lens in such a manner that at least one of the IR μLEDs,which is contained inside of a peripheral edge boundary of theillumination lens, is positioned in a user line-of-sight area thatpasses through the illumination lens (act 910), as described above.

Next, the HMD is also configured to emit IR light from one or more ofthe IR μLEDs in the illumination lens towards an eye of the user duringthe use of the HMD (act 920), based on the positioning of theillumination lens on the HMD and by controlled illumination of the IRμLEDs by the system components described below in reference to FIG. 11 ,for example.

Next, the HMD is further configured to detect and process glints of theIR light that is reflected back from the user's eye during the use ofthe HMD (act 930) and to determine a positioning of the user's eye basedon the detected and processed glints (act 940). The system componentsdescribed below in reference to FIG. 11 can also be used to perform thisfunctionality, such as, for example, the disclosed I/O interfaces 1120,camera(s)/sensor(s) 1130, illumination control module 1150, lightprocessing module 1160, processor(s) 1120 and code 1170.

Attention will now be directed to FIG. 10 , which illustrates a flowdiagram 1000 of acts associated with methods for manufacturing anillumination lens structure with a plurality of IR μLEDs (infrared microlight emitting devices). As shown, the acts include an act for obtaininga transparent backplane or substrate (act 1010). Such a substrate 820 orbackplane is illustrated and described in reference to FIG. 8 .

The illustrated acts also include applying a plurality of traces to thetransparent backplane (act 1020), wherein the plurality of traces iselectrically conductive and forming at least one electrical circuitbetween an anode terminal and a cathode terminal. Various knowntechniques for forming the traces can be utilized, include variousdeposition and etching processes known to those of skill in the art.

The illustrated acts also include the obtaining of an IR μLED wafer (act1030), such as the epitaxy wafer 810 referenced in FIG. 8 , whichcontains a plurality of IR μLEDs or material that can be separatelyextracted as discrete IR μLEDs having a maximum dimension of <100 μm.

Next, a set of one or more of the IR μLEDs are transferred to thesubstrate/backplane (act 1040), directly on the traces/circuits of thetransparent backplane/substrate, in a predetermined pattern, and suchthat the IR μLEDs are electrically coupled to at least one electricalcircuit on the backplane/substrate.

In some instances, a segment of the transparent backplane is alsoseparated from a remaining portion of the transparent backplane (act1060), thereby forming a stand-alone illumination lens structure thatincludes at least one electrical circuit with the referenced set of oneor more IR μLEDs.

The disclosed embodiments also include applying one or more protectivecoating to the illumination lens structure before or after it isseparated from the rest of the transparent backplane.

Although not illustrated, the disclosed acts can also includeelectrically and mechanically coupling the illumination lens structureto a HMD (head mounted display) in such a manner as to be configured toemit IR light for facilitating eye tracking of a user's eyes by the HMD,the HMD being configured to perform the eye tracking based on detectingand processing IR light reflected off of the user's eyes that is emittedfrom the set of one or more IR μLEDs.

Attention will now be directed to FIG. 11 , which illustrates anexemplary computing system 1100 that can incorporate and/or be used toimplement the disclosed embodiments. As used herein, “computer system,”“computing system,” and simply “computer” are similar terms that may beinterchanged with each other. Further, the computer system 1100 may takeany form, such as, for example, an HMD.

As illustrated, the computer system 1100 includes at least one hardwareprocessing unit 1110 (aka “processor(s)”), input/output (I/O) interfaces1120, one or more sensors 1130 (e.g., eye tracking cameras and sensors),and storage 1140. The computer system 1100 also includes variousdifferent components that are useful for tracking a user's eye. Toillustrate, the computer system 110 includes an illumination controlmodule 1150 and a light processing module 1160. More detail on thesecomponents will be discussed later. Although not shown, the system 1100may also include graphics rendering engines for rendering images and apower supply for supplying the different components described herein.

The illustrated storage 1140 may be physical system memory, which may bevolatile, non-volatile, or some combination of the two. As such, thestorage 1140 may be considered a computer-readable hardware storagedevice that is capable of storing computer-executable instructions(e.g., “code” 1170) that are executable by the processor(s) 1110 toconfigure the computing system 1100 to implement some of thefunctionality described herein, including the referenced eye trackingfunctionality.

In particular, execution of the code 1170 can cause the illuminationcontrol module 1150 to activate/power the IR μLEDs/circuits referencedabove, to cause IR light to illuminate a user's eye. The execution ofthe code 1170 can also cause the camera(s)/sensor(s) 1130 to capture andmeasure the IR light reflections/glints that are reflected off of auser's eye during use and during illumination by the IR μLEDs.Furthermore, The execution of the code 1170 can also cause the lightprocessing module 1160 to convert the measured IR light signals detectedby the camera(s) into identifiable position mappings of the user'seye(s).

The various I/O interfaces 1120 are configured to interconnect thevarious other system components and to also provide interfaces forenabling a user to control the settings and operation of the differentsystem components.

The referenced computer-executable (or computer-interpretable)instructions that are stored as code 1170, and which are executable bythe processor(s) comprise, for example, instructions that cause ageneral-purpose computer, special-purpose computer, or special-purposeprocessing device to perform a certain function or group of functions.The computer-executable instructions may be, for example, binaries,intermediate format instructions such as assembly language, or evensource code. Although the subject matter has been described in languagespecific to structural features and/or methodological acts, it is to beunderstood that the subject matter defined in the appended claims is notnecessarily limited to the described features or acts described above.Rather, the described features and acts are disclosed as example formsof implementing the claims.

Those skilled in the art will appreciate that the embodiments may bepracticed in network computing environments with many types of computersystem configurations, including personal computers, desktop computers,laptop computers, message processors, hand-held devices, multi-processorsystems, microprocessor-based or programmable consumer electronics,network PCs, minicomputers, mainframe computers, mobile telephones,PDAs, pagers, routers, switches, and the like. The embodiments may alsobe practiced in distributed system environments where local and remotecomputer systems that are linked (either by hardwired data links,wireless data links, or by a combination of hardwired and wireless datalinks) through a network each perform tasks (e.g. cloud computing, cloudservices and the like). In a distributed system environment, the varioussystem components and code may be located in both local and remotememory storage devices.

Additionally or alternatively, the functionality described herein can beperformed, at least in part, by one or more hardware logic components.For example, and without limitation, illustrative types of hardwarelogic components that can be used include Field-Programmable Gate Arrays(FPGAs), Program-Specific or Application-Specific Integrated Circuits(ASICs), Program-Specific Standard Products (ASSPs), System-On-A-ChipSystems (SOCs), Complex Programmable Logic Devices (CPLDs), CentralProcessing Units (CPUs), and other types of programmable hardware.

By practicing the principles disclosed herein, significant advantagesmay be realized, including, but not limited to, the creation and use oftransparent illumination lens structures having a plurality of IR μLEDsdistributed within the transparent viewing area of the illumination lensstructures and that are small enough (<100 μm) that they are not visibleby a user during use of an HMD or other mixed-reality device thatincorporates the transparent illumination lens structures forilluminating a user's eyes during use for enabling eye tracking andwithout visibly obscuring or interfering with the user's view of amixed-reality environment by the mixed-reality device through thetransparent illumination lens structures.

The present invention may be embodied in other specific forms withoutdeparting from its spirit or essential characteristics. The describedembodiments are to be considered in all respects only as illustrativeand not restrictive. The scope of the invention is, therefore, indicatedby the appended claims rather than by the foregoing description. Allchanges which come within the meaning and range of equivalency of theclaims are to be embraced within their scope.

What is claimed is:
 1. A method of manufacturing a transparentillumination structure with a plurality of IR μLEDs (infrared microlight emitting devices), the method comprising: obtaining a transparentbackplane; applying a plurality of traces to the transparent backplane,the plurality of traces being electrically conductive and forming atleast one electrical circuit between an anode terminal and a cathodeterminal; obtain a wafer containing a plurality of IR μLEDs; andtransfer a set of one or more IR μLEDs from the wafer onto thetransparent backplane in a predetermined pattern, the set of one or moreIR μLEDs being positioned directly onto the traces, IR μLED in the setof one or more IR μLEDs being electrically coupled with the at least oneelectrical circuit.
 2. The method of claim 1, wherein the method furtherincludes: applying a protective coating over the set of one or more IRμLEDs.
 3. The method of claim 1, wherein the predetermined patter is acircular pattern.
 4. The method of claim 1, wherein the predeterminedpatter is a rectangular pattern.
 5. The method of claim 1, wherein thepredetermined patter is a diamond pattern.
 6. The method of claim 1,wherein the set of one or more IR μLEDs comprises a quantity of IR μLEDsbetween two and twenty.
 7. The method of claim 1, wherein the set of oneor more IR μLEDs comprises a quantity of IR μLEDs between three andtwelve.
 8. The method of claim 1, wherein each IR μLED of the set of oneor more IR μLEDs is configured to only emit light having wavelengthsabove 800 nm.
 9. The method of claim 1, wherein each IR μLED of the setof one or more IR μLEDs is configured to emit light having a wavelengthof about 850 nm.
 10. The method of claim 1, wherein the method furtherincludes: separating a segment of the transparent backplane from aremaining portion of the transparent backplane, the segment of thetransparent backplane including the least one electrical circuit withthe set of one or more IR μLEDs.
 11. The method of claim 10, wherein themethod further includes electrically and mechanically coupling thesegment of the transparent backplane to a HMD (head mounted display) asan illumination lens, the illumination lens being configured to emit IRlight for facilitating eye tracking of a user's eyes by the HMD, the HMDbeing configured to perform the eye tracking based on detecting andprocessing IR light reflected off of the user's eyes that is emittedfrom the set of one or more IR μLEDs.
 12. The method of claim 11,wherein the set of one or more IR μLEDs are positioned within theillumination lens in such a configuration that at least some IR μLEDs ofthe plurality of IR μLEDs are within a line-of-sight by the user whenthe user wears the HMD during use.
 13. The method of claim 12, whereinthe at least some IR μLEDs of the plurality of IR μLEDs have a maximumsize dimension of <100 μm.
 14. The method of claim 1, wherein thetransparent backplane is a rigid glass structure.
 15. The method ofclaim 1, wherein the transparent backplane is a flexible plasticstructure.
 16. An HMD configured to perform eye tracking, the HMDcomprising: an illumination display component that contains a pluralityof IR μLEDs having a pattern of distribution on or within theillumination display component inside of a peripheral edge boundary ofthe illumination display component, with at least one IR μLED of theplurality of IR μLEDs being positioned in the pattern within a userline-of-sight area passing through the illumination display component; acamera sensor configured to detect IR light emitted from the at leastone IR μLED and that is reflected from a user's eye during use of theHMD; and a light processing module configured to identify positioning ofthe user's eye based on processing signals generated from the detectedIR light that is reflected from the user's eye during use of the HMD.17. The HMD recited in claim 16, wherein the HMD is further configuredto generate and render one or more virtual objects to the user, duringuse of the HMD, based at least in part on the identified positioning ofthe user's eye.
 18. The HMD recited in claim 16, wherein the HMD isfurther configured with an illumination control for selectively andcontrollably illuminating said at least one of the IR μLED separatelyfrom one or more other IR μLED of the plurality of μLEDs.
 19. The HMDrecited in claim 16, wherein the predetermined of distribution is acircular pattern.
 20. The HMD recited in claim 16, wherein at least oneof the IR μLED of the plurality of IR μLEDs has a maximum size dimensionof <100 μm.